In recent years, studies about mixed reality that aims at seamless joint of real and virtual spaces have been extensively made. The studies about mixed reality aim at coexistence of a virtual reality world which can be conventionally experienced in only a situation isolated from the real space, and the real space, and have attracted attention as a technique that augments virtual reality. As applications of such mixed reality, new fields different in quality from conventional virtual reality such as a medical assistant application that presents the state in the body of a patient to a doctor as if it were seen through, an operation assistant application that superimposes the assembling order of a product on real objects in a factory, and the like are expected.
A video see-through AR (Augmented Reality) system as one of presentation methods of such mixed reality generates a mixed reality image by superposing and compositing a real space image obtained by capturing a real space in real time using a camera or the like, and a virtual space image which has the same field angle as that of the camera and is obtained by a computer, and displays the generated mixed reality image on an image display device.
Some conventional techniques, which use a camera whose zoom ratio can be optically changed as the camera for capturing the real space in this system, have been disclosed (for example, see Japanese Patent Laid-Open Nos. 6-105232 and 2001-285715, and Japanese Patent Nos. 3,122,099, 3,074,943, and 3,247,307).
A change in zoom ratio of the camera changes the photographing field angle (=focal length). Therefore, in the above prior arts, the photographing field angle of the camera used to sense a real space is acquired in real time, and is used upon generating a virtual space image, so as to match the field angle of the camera used to sense the real space image and that used upon generating a virtual space image. As acquisition means of the field angle of the camera, it is a common practice to measure the lens state such as a rotation angle or the like of a zoom lens and to acquire the field angle of the camera according to a predetermined conversion model based on the measured value.
Note that the above prior arts implement the mixed reality presentation method using the camera whose zoom ratio can be changed under the condition that the field angle of the camera acquired by the acquisition means is correct. However, the acquired field angle of the camera does not always match that of an actual camera due to measurement errors of the lens state, errors of the conversion model from the lens state of the photographing field angle of the camera, and the like.
In the video see-through AR system, the size of the real space image sensed by the camera may change when the real space image is captured into a computer and is converted into digital data. Hence, the field angle of an image sensed by the camera is slightly different from that of the real space image to be composited with the virtual space image. Since a change in size independently takes place in the vertical and horizontal directions of the image, the images may have different field angles in their vertical and horizontal directions. When such field angle difference is not negligible, conversion from the lens state into the field angle of the real space image to be composited must be done in place of conversion from the lens state into the photographing field angle of the camera. However, it is difficult to model this conversion.